Jetpack Search AI Answers: add dashboard and front end ui. by gibrown · Pull Request #48592 · Automattic/jetpack

gibrown · 2026-05-07T02:48:53Z

This is a squashed commit from developing the feature and multiple experiments with the interface in #48251

Proposed changes

Add an AI powered search answers to the existing search modal.

Customizable personality and behavior for the AI
Summarize search results and answer questions directly
Fills in the gap when there are no search results by relaxing the search query
Launch for all paid subscribers
New dashboard tab for customizing and enabling the feature.

Testing

Basics

Start up local Jetpack site in docker and connect via jurassic tube so that it is a connected site.
Add a site override so you can search content from a site with decent content. Any public site should work. This example uses jetpack.com (which is jetpackme.wordpress.com)

tools/docker/mu-plugins/jetpack-search-site-override.php

/**
 * Override the Search overlay site ID for local development.
 * Allows testing against a specific site's content (e.g. jetpack.com).
 */
add_filter( 'jetpack_instant_search_options', function ( $options ) {
	$options['siteId']          = 20115252; // jetpack.com
	$options['aiAnswersSiteId'] = 20115252; // jetpack.com
	$options['homeUrl']         = 'jetpackme.wordpress.com';
	return $options;
} );

Run some search queries on the site with just /?s=search

Testing errors

To test out errors use the following in your browser console:

Network error (simplest — just block it in the Network tab too):

  const _fetch = window.fetch;
  window.fetch = (url, opts) => url.includes('ai/agent')
    ? Promise.reject(new TypeError('Failed to fetch'))
    : _fetch(url, opts);

HTTP error (e.g. 503):

  const _fetch = window.fetch;
  window.fetch = (url, opts) => url.includes('ai/agent')
    ? Promise.resolve(new Response('', { status: 503 }))
    : _fetch(url, opts);

API-level task failure (SSE stream returns the error JSON):

  const _fetch = window.fetch;
  window.fetch = (url, opts) => {
    if (!url.includes('ai/agent')) return _fetch(url, opts);
    const errorEvent = `data: ${JSON.stringify({
      jsonrpc: '2.0', id: 'req-1',
      result: {
        type: 'TaskStatusUpdateEvent',
        status: { state: 'failed', message: { parts: [{ type: 'text', text: 'Task failed on server' }] } }
      },
      error: { code: -32000, message: 'An error occurred while processing the request. Please try again later.' }
    })}\n\n`;
    const stream = new ReadableStream({
      start(c) { c.enqueue(new TextEncoder().encode(errorEvent)); c.close(); }
    });
    return Promise.resolve(new Response(stream, {
      status: 200, headers: { 'Content-Type': 'text/event-stream' }
    }));
  };

Restore normal behaviour:

  window.fetch = _fetch;

Testing on fieldguide or P2

Sandbox the sites you are interested in testing.
Run this command on your sandbox

~/public_html/bin/jetpack-downloader test jetpack jps3-ai-answers

Enable the AI answers option in the jetpack search settings in wp-admin, e.g. http:///wp-admin/admin.php?page=jetpack-search
To test, you must be logged into wordpress.com. I recommend doing this in one tab and then opening the sandboxed site in another tab. You will likely get an SSL/HSTS error. If using Chrome, you can click on 'advanced options' and then type "thisisunsafe". There are other alternatives as well. Search on the Field Guide for "ssl chrome".

Try asking some queries and see what you think of the AI answers.

Turning on the new JPS experience selection in the dashboard

Add into a mu-plugins files:

add_filter( 'jetpack_search_blocks_enabled', '__return_true' );

github-actions · 2026-05-07T02:49:56Z

Thank you for your PR!

When contributing to Jetpack, we have a few suggestions that can help us test and review your patch:

✅ Include a description of your PR changes.
✅ Add a "[Status]" label (In Progress, Needs Review, ...).
🔴 Add testing instructions.
🔴 Specify whether this PR includes any changes to data or privacy.
✅ Add changelog entries to affected projects

This comment will be updated as you work on your PR and make changes. If you think that some of those checks are not needed for your PR, please explain why you think so. Thanks for cooperation 🤖

🔴 Action required: Please include detailed testing steps, explaining how to test your change, like so:

## Testing instructions:

* Go to '..'
*

🔴 Action required: We would recommend that you add a section to the PR description to specify whether this PR includes any changes to data or privacy, like so:

## Does this pull request change what data or activity we track or use?

My PR adds *x* and *y*.

Follow this PR Review Process:

Ensure all required checks appearing at the bottom of this PR are passing.
Make sure to test your changes on all platforms that it applies to. You're responsible for the quality of the code you ship.
You can use GitHub's Reviewers functionality to request a review.
When it's reviewed and merged, you will be pinged in Slack to deploy the changes to WordPress.com simple once the build is done.

If you have questions about anything, reach out in #jetpack-developers for guidance!

github-actions · 2026-05-07T04:42:13Z

Are you an Automattician? Please test your changes on all WordPress.com environments to help mitigate accidental explosions.

To test on WoA, go to the Plugins menu on a WoA dev site. Click on the "Upload" button and follow the upgrade flow to be able to upload, install, and activate the Jetpack Beta plugin. Once the plugin is active, go to Jetpack > Jetpack Beta, select your plugin (Jetpack or WordPress.com Site Helper), and enable the jps3-ai-answers branch.
To test on Simple, run the following command on your sandbox:

bin/jetpack-downloader test jetpack jps3-ai-answers

bin/jetpack-downloader test jetpack-mu-wpcom-plugin jps3-ai-answers

Interested in more tips and information?

In your local development environment, use the jetpack rsync command to sync your changes to a WoA dev blog.
Read more about our development workflow here: PCYsg-eg0-p2
Figure out when your changes will be shipped to customers here: PCYsg-eg5-p2

jp-launch-control · 2026-05-07T04:57:39Z

Code Coverage Summary

Coverage changed in 17 files. Only the first 5 are listed here.

File	Coverage	Δ%	Δ Uncovered
projects/packages/search/src/class-settings.php	0/31 (0.00%)	0.00%	1 ❤️‍🩹
projects/packages/search/src/class-helper.php	313/378 (82.80%)	0.05%	0 💚
projects/packages/search/src/class-rest-controller.php	215/252 (85.32%)	0.36%	0 💚
projects/packages/search/src/dashboard/components/pages/dashboard-page.jsx	47/56 (83.93%)	0.29%	0 💚
projects/packages/search/src/dashboard/store/selectors/jetpack-settings.js	26/26 (100.00%)	0.00%	0 💚

7 files are newly checked for coverage. Only the first 5 are listed here.

File	Coverage
projects/packages/search/src/instant-search/lib/markdown.js	32/60 (53.33%) 💚
projects/packages/search/src/dashboard/hooks/use-search-settings.js	6/7 (85.71%) 💚
projects/packages/search/src/instant-search/components/animated-ellipsis.jsx	9/10 (90.00%) 💚
projects/packages/search/src/class-ai-answers.php	41/43 (95.35%) 💚
projects/packages/search/src/instant-search/components/answers-panel.jsx	23/24 (95.83%) 💚

Full summary · PHP report · JS report

jjolmo · 2026-05-08T15:35:15Z

I've did just a quick round and smoke test

It is working. I tested also with wrong keywords, the result is expected (No results found)

Some quick smoke test:

Basics

Case	Status
Dashboard tab "AI Answers (Preview)"	✅ Appears after plan upgrade
"Enable AI Answers" toggle	✅ Works and persists
AI ANSWER panel in search overlay	✅ Renders above results
Response streaming	✅ Text appears progressively
"Show more" → expanded response	✅ Setup Steps + Key Features + Sources
Citation tags at the bottom	✅ 5 clickable sources
Fill-the-gap (no results found)	✅ AI still answers when search returns 0

Testing errors

Case	Status
Network error (`Failed to fetch`)	✅ "Sorry, an error occurred while generating an answer."
HTTP error (503)	✅ Generic error + sub-text "Service Unavailable"
API-level task failure (SSE error JSON)	✅ Generic error + `error.message` + `Error code: -32000`
Restore normal fetch	✅ Live answers resume immediately

jjolmo · 2026-05-08T15:37:04Z

🤖 Minor security nit (defense-in-depth, not a blocker):

Citation URLs from the SSE stream are rendered as <a href={ url }> in two places without scheme validation:

projects/packages/search/src/instant-search/components/answers-panel.jsx:153
projects/packages/search/src/instant-search/components/sidebar.jsx:16

React doesn't strip javascript: URLs in href, so a tainted sources[].url from the agent (e.g. via prompt injection in indexed content) would XSS on click. The markdown body is already protected by the ^https?:// allowlist in markdown.js — would be nice to mirror that for citations:

const safeUrl = /^https?:\/\//i.test( citation.url ) ? citation.url : '#';

Also worth wrapping new URL(citation.url).hostname in a try/catch in sidebar.jsx so a malformed URL doesn't crash the panel.

Update — PoC reproduced locally

Hijacked fetch to the AI agent endpoint and forged a completed SSE event with a poisoned citation URL:

sources: [{
  title: '👉 PoC: javascript: URL injected as citation',
  url: "javascript:alert('XSS — origin '+document.domain)"
}]

The href round-tripped intact through React render:

[{
  "text": "PoC: javascript: URL injected as citation",
  "href": "javascript:alert('XSS — origin '+document.domain)"
}]

On activation (browser-faithful eval of the href, equivalent to a real click):

{
  "executedOn": "PoC: javascript: URL injected as citation",
  "captured": ["XSS — origin javijetpack.jurassic.tube"]
}

Code executed in the site origin with full access to document.cookie, wp-admin session, and the wpcom REST proxy. Same vector applies to the sidebar.jsx citations block.

robfelty · 2026-05-08T20:33:10Z

Thanks for the testing @jjolmo - and for the security issue too!

kangzj · 2026-05-11T03:28:15Z

Hi folks, Just a gentle nudge for the PR to land if everything is looking good now. We are almost ready to launch the Search Blocks, and Dashboard is the final piece now. Let us know if there's anything we could do to help 🙂

cc/ @adamwoodnz

jjolmo

I see it in DRAFT so I forgot to approve. But here's my LGTM

kangzj · 2026-05-11T08:52:53Z

BTW we should probably strip/simplify the docs before merging.

robfelty · 2026-05-11T12:19:03Z

I am working on resolving merge conflicts and one last update. Then I think we should be ready to go.

robfelty

Did lots of testing. I think it is good enough to ship and then do small iterations.

This is a squashed commit from developing the feature and multiple experiments with the interface. It also includes the original plan for building the feature. This was merged with the new experience selection logic from #48500

…dmap.

… a few things.

Rebased from trunk

Drop redundant package-name prefixes inside each package's own changelog (per the skill style guide § Prefixes and scope), tighten the search package's Added entries the reviewer flagged as non-imperative or too technical, split compound bullets, and regenerate the Jetpack readme: - charts, components, activity-log, forms, search (Fixed), jetpack plugin (Improved compatibility): drop the prefix that repeats the package name. - search Added: rephrase the REST entry to start with `Add` and drop the `Module_Control` aside; de-internalize the rebuild-mode entry (no `Custom_Taxonomy_Slot_Mapping::backfill()`); split the Results List auto-switch entry into the behavior bullet and the opt-out toggle bullet (#48943); capitalize Whitelist; drop the `Search:` prefix on the WP-CLI backfill entry; split the `Search: AI Answers` entry into two `AI Answers:` bullets — one for the streaming panel, one for the customization dashboard (#48592). - Regenerated `projects/plugins/jetpack/readme.txt` via `tools/plugin-changelog-to-readme.sh jetpack`.

gibrown requested a review from robfelty May 7, 2026 02:48

gibrown self-assigned this May 7, 2026

gibrown added [Status] In Progress [Feature] Search For all things related to Search labels May 7, 2026

github-actions Bot added [Package] Search Contains core Search functionality for Jetpack and Search plugins [Package] Sync [Tests] Includes Tests Docs E2E Tests labels May 7, 2026

gibrown force-pushed the jps3-ai-answers branch from ebe1878 to ec26f9e Compare May 7, 2026 04:41

gibrown mentioned this pull request May 7, 2026

Jetpack Search: Add Answers UI #48251

Closed

robfelty force-pushed the jps3-ai-answers branch from 323ad09 to 1665d5b Compare May 8, 2026 21:41

jjolmo self-requested a review May 11, 2026 07:25

jjolmo previously approved these changes May 11, 2026

View reviewed changes

robfelty marked this pull request as ready for review May 11, 2026 12:19

robfelty requested a review from a team as a code owner May 11, 2026 12:19

robfelty dismissed jjolmo’s stale review via 43583f0 May 11, 2026 12:32

robfelty force-pushed the jps3-ai-answers branch from 1665d5b to 43583f0 Compare May 11, 2026 12:32

robfelty previously approved these changes May 11, 2026

View reviewed changes

robfelty dismissed their stale review via becc955 May 11, 2026 23:41

gibrown and others added 12 commits May 11, 2026 19:58

Add search tests to improve test coverage.

186514b

Further improve test coverage and clean up the changelog file.

434f7db

Clean up old planning and design docs from the branch. Update the roa…

e03d83d

…dmap.

Address PR feedback: clean up css, some minor oversights, and punt on…

a3bfb64

… a few things.

Fix mistake with the changelogs.

e426393

Updated REST API calls to use new combined workflow

03e4305

Addressed possible security concern

e1bf0a8

No longer requiring Gutenberg

cc28ad1

Rebased from trunk

Fixed tests

e0f6753

fixed tests

8afdb7c

static analysis fix

ae7e23e

Getting rid of unnecessary token

840393f

robfelty force-pushed the jps3-ai-answers branch from becc955 to 840393f Compare May 12, 2026 00:24

robfelty added 3 commits May 11, 2026 21:00

Fixed issue with saving personality

56e5a24

fixed personality settings

fc9f5f4

another attempt at fixing saving personality

fbf1794

robfelty merged commit 396bb85 into trunk May 12, 2026
92 checks passed

robfelty deleted the jps3-ai-answers branch May 12, 2026 14:07

github-actions Bot added [Status] UI Changes Add this to PRs that change the UI so documentation can be updated. and removed [Status] In Progress labels May 12, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Jetpack Search AI Answers: add dashboard and front end ui.#48592

Jetpack Search AI Answers: add dashboard and front end ui.#48592
robfelty merged 16 commits into
trunkfrom
jps3-ai-answers

gibrown commented May 7, 2026 •

edited by robfelty

Loading

Uh oh!

github-actions Bot commented May 7, 2026

Uh oh!

github-actions Bot commented May 7, 2026 •

edited

Loading

Uh oh!

jp-launch-control Bot commented May 7, 2026 •

edited

Loading

Uh oh!

jjolmo commented May 8, 2026

Uh oh!

jjolmo commented May 8, 2026 •

edited

Loading

Uh oh!

robfelty commented May 8, 2026

Uh oh!

kangzj commented May 11, 2026

Uh oh!

jjolmo left a comment

Uh oh!

kangzj commented May 11, 2026 •

edited

Loading

Uh oh!

robfelty commented May 11, 2026

Uh oh!

robfelty left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

gibrown commented May 7, 2026 • edited by robfelty Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed changes

Testing

Basics

Testing errors

Testing on fieldguide or P2

Turning on the new JPS experience selection in the dashboard

Uh oh!

github-actions Bot commented May 7, 2026

Uh oh!

github-actions Bot commented May 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jp-launch-control Bot commented May 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Coverage Summary

Uh oh!

jjolmo commented May 8, 2026

Basics

Testing errors

Uh oh!

jjolmo commented May 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Update — PoC reproduced locally

Uh oh!

robfelty commented May 8, 2026

Uh oh!

kangzj commented May 11, 2026

Uh oh!

jjolmo left a comment

Choose a reason for hiding this comment

Uh oh!

kangzj commented May 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

robfelty commented May 11, 2026

Uh oh!

robfelty left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

gibrown commented May 7, 2026 •

edited by robfelty

Loading

github-actions Bot commented May 7, 2026 •

edited

Loading

jp-launch-control Bot commented May 7, 2026 •

edited

Loading

jjolmo commented May 8, 2026 •

edited

Loading

kangzj commented May 11, 2026 •

edited

Loading